REMARKS 

Claims 1-7 have been amended, and new claims 8-14 have been added. 
Claims 1-14 are pending, with claims 1, 6, and 12 being independent. 

Attached hereto is an Appendix entitled "Version with Markings to Show 
Changes Made" which is a marked-up version of the portions of the application 
which have been amended by the present amendment, with brackets indicating 
deleted matter and underlining indicating added matter. 

In item 13 on page 1 (the Office Action Summary) of the Office Action of 
August 1, 2002, the Examiner has checked boxes J_3 and al in an attempt to 
acknowledge the applicants' claim for foreign priority and acknowledge that the 
certified copy of the priority document has been received. However, the 
Examiner did not check box 1. Certified copies of the priority doc uments have 
been received, as the Examiner was supposed to do to make a complete 
acknowledgement that the certified copy of the priority document has been 
received. Accordingly, it is respectfully requested that the Examiner make a 
complete acknowledge by checking boxes 1_3, air and 1 . Certified copies of the 
priority documents have been received, in the next Office communication. 

A copy of Japanese reference 10-304235 (including an English abstract) 
which is cited on page 2 of the specification was submitted on November 21, 
2001, when the present application was filed, together with a sheet entitled 
"Information under 37 CFR 1.56(a) listing this reference. The Office Action of 
August 1, 2002, includes a copy of this sheet marked by the Examiner to 
indicate that Japanese reference 10-304235 has been considered. However, 
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sometimes the PTO does not print references listed on such a sheet filed in an 
application in the "References Cited" section of a patent issuing from the 
application. 

Accordingly, to ensure that Japanese reference 10-304235 will be listed 
j n t he "References Cited" section of any patent which may issue from the 
present application, attached hereto is a form PTO-1449 listing Japanese 
reference 10-304235. It is respectfully requested that the Examiner mark the 
form PTO-1449 to indicate that Japanese reference 10-304235 has been 
considered, and provide a copy of the marked form PTO-1449 with the next 
Office communication. 

Submitted herewith is a proposed correction to Fig. 3. Corrected formal 
drawings including the change in this proposed correction will be submitted 
when required by the Examiner. 

The drawings were objected to because of a spelling error in step 3 in Fig. 
3. The applicants have proposed to correct this error in the proposed drawing 
correction submitted herewith, and accordingly it is respectfully requested that 
the objection to the drawings be withdrawn . 

Claims 1-7 have been amended solely to improve their form. No new 
limitations have intentionally been added to claims 1-7 in an at tempt to 
distinguish them over the prior art. In light of this, it is submitted that it would 
not be proper for the Examiner to make the next Office Action final if it includes 
any new ground of rejection of any of claims 1-7 over the prior art. 
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New dependent claims 8-11 depending from independent claims 1 and 6 
have been added to recite further combinations of features of the present 
invention. 

New independent claim 12 and new dependent claims 13-14 depending 
therefrom have been added to recite the present invention in different terms. 

Claims 1-7 were rejected under 35 USC 102(a) as being anticipated by 
Howell (U.S. Patent No. 6,046,772). This rejection is respectfully traversed, 
both with respect to claims 1-7 and insofar as the rejection may be deemed to 
be applicable to new claims 8-14. 

Independent claim 1 recites an image processing apparatus comprising an 
imaging optical system for forming an image of an object on an imaging surface, 
a color imaging device including a plurality of photo detectors arranged on the 
imaging surface, each of the photo detectors forming a pixel, and a plurality of 
sets of four filters for three colors, two filters of the four filters being for a 
selected color of the three colors, the filters being arranged at positions 
respectively corresponding to the photo detectors, the image of the object being 
formed on the photo detectors through the filters by the imaging optical system, 
shift drive means for shifting the imaging optical system and the photo detectors 
relative to each other by a distance corresponding to a predetermined number of 
pixels on the imaging surface, and an image processing unit for generating an 
image using a plurality of image data picked up before and after the shifting, 
wherein the image processing unit generates and outputs a single 
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monochromatic image using only pixel data detected bv photo detectors 

corresponding to the two filters for the selected color of the th ree colors. 

In explaining the rejection of claim 1, the Examiner states as follows: 

In regards to claim 1 Howell et al, herein Howell, 
discloses all of the necessary information in order to 
reject all parts of the claimed "image processing 
apparatus" as noted in the following five paragraphs. 

In regards to the claimed "imaging optical system 
for forming an image of an object on an imaging 
surface" as claimed in claim 1 Howell discloses a 
digital camera that has "at the focal plane an array of 
electronic light-sensitive elements ... that produce a 
light-intensity-dependant electrical signal in response 
to being illuminated (column 1, lines 29-33, Howell)." 

In regards to the claimed "color imaging device" as 
claimed in claim 1 Howell discloses an "array of 
electronic light-sensitive elements (column 1, line 30, 
Howell)." Howell further discloses "the color matrix in 
the present invention is based on a unit cell of primary 
colors that is a mosaic in which half the pixels are one 
primary color and the rest are divided equally between 
the two complementary primary colors (column 7, 
lines 14-18, Howell)." Note figure 3 depicting a 
"multiplicity of sets of four filters for three colors, two 
of said four filters being used for selected one of said 
three colors." Howell still further discloses "the 
unaided light-sensing elements do not distinguish 
between different colors of light ... In order to capture 
the color data needed ... one can use several primary 
color filters (column 2, lines 28-32, Howell)," or in 
other words the "object image being formed on said 
photo detectors through each of said filters by said 
imaging optical system." 

In regards to the claimed "shift drive means for 
shifting said imaging optical system and said photo 
detectors relatively to each other by a length 
corresponding to a predetermined number of pixels in 
said imaging surface" as claimed in claim 1 Howell 
discloses "there are means provided in many digital 
cameras currently in use by which the light-sensitive 
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array can be shifted by a single pixel ... the present 
invention utilizes a precision rotation of the digital 
camera about an axis passing through the principal 
point of its lens (column 9, lines 28-35, Howell)," 
whereby the "predetermined number of pixels" is "a 
single pixel (column 9, line 30, Howell)." 

In regards to the claimed "image processing unit for 
generating an image using a plurality of image data 
picked up before and after said shift" as claimed in 
claim 1 Howell discloses "two photographs will serve 
to record the image in one primary color for every 
pixel position (column 7, lines 25-26, Howell)." 
Howell further discloses "between the two 
photographs the light sensing array is shifted with 
respect to the image by one pixel (column 7, lines 
27-29, Howell)." Howell still further discloses 
"electronic circuitry/software used to combine the two 
images produced in the two-shot mode uses the 
luminous intensity of a particular primary color at all 
locations in the image to produce an 
integrated/interpolated image (column 7, lines 31-35, 
Howell)." 

In regards to the claimed "image processing unit 
generates and outputs a single monochromatic image 
using only the pixel data detected by the photo 
detectors having said two filters of a single color" as 
claimed in claim 1 Howell discloses "as one records," 
or "outputs," " a series of photographs; each image," 
or "single ... image," captured will depict the scene as 
it appears in one primary color (column 2, line 35, 
Howell)," or "monochromatic," wherein one of the 
"series of photographs (column 2, line 33, Howell)" is 
of the color green as disclosed in figure 6. Howell 
further discloses from figure 6 that "the result of 
having taken the two single shots will be to obtain 
actual measurement data for green for every single 
point (column 9, lines 8-10, Howell)," or a "single 
monochromatic image" in green wherein the green 
color filter in figure 6 is the same as the claimed "said 
two filters of a single color." 

Thus, the Examiner considers column 2, lines 33 and 35; column 9, lines 

9-10; and Fig. 6 of Howell to disclose the feature of claim 1 wherein the image 
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processing unit generates and outputs a single monochr omatic image using only 
pixel data detected bv photo detectors corresponding to the two filters for the 
selected color of the three colors . 

Column 2, lines 33 and 35, of Howell referred to by the Examiner is part 
of the longer passage in column 2, lines 30-41, of Howell which reads as 
follows (emphasis added): 

In order to capture the color data needed for 
reconstituting the image in color, one can use several 
primary color filters in succession as one records a 
series of photographs; each image captured will depict 
the scene as it appears in one primary color. Then, 
these images — each in one primary color — can be 
combined using electronic circuitry, typically 
software-controlled, to produce and display a final, 
composite imaoe that is reasonably true in color 
distribution to the image that would be formed on the 
focal plane without anv intervening color filters , i.e., a 
final image reasonably true to the colors of the scene 
being photographed. 

While this passage of Howell refers to several captured images eac h being 
in a single primary color , it is submitted that Howell does not disclose an image 
processing unit which generates and outputs a single m onochromatic image 
using only pixel data detected bv photo detectors corresp onding to the two 
filters for the selected color of the three colors as recited in claim 1 . 

This passage of Howell refers to electronic circuitry, typically 
software-controlled . Assuming arguendo that Howell's electronic circuitry 
corresponds to an image processing unit as recited in claim 1, it is submitted 
that Howell's electronic circuitry generates and outpu ts a single color image 
using pixel data detected by photo detectors corresponding to fil ters for all of 
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three colors , rather than generating and outouttinq a single mo nochromatic 
image using only pixel data detected bv photo detectors c orresponding to the 
two filters for the selected color of the three colors as recited in claim 1 . 

Column 9, lines 9-10, of Howell referred to by the Examiner refers to Fig. 
6 of Howell referred to by the Examiner, and is part of the longer passage in 
column 8, line 66, through column 9, line 27, of Howell which reads as follows 
(emphasis added): 

FIG. 6 shows the data that the camera of the 
Preferred Embodiment of the present invention 
captures in two-shot operation. The positions of the 
array for the two shots are shown, respectively, by 
the two arrays at the top of FIG. 6, the ones joined by 
an addition sign (the 8x8 arrays being intended to 
represent an internal sub-array of the actual array, 
itself hundreds or even thousands of pixels on a side). 
The array on the right can be seen to result from a 
one-pixel shift to the right (or to the left) of the array 
on the left. 

As is emphasized in the middle arrays of FIG. 6, the 
result of having taken the two single shots will be to 
obtain actual measurement data for green for every 
single point . Moreover, as shown by the composite 
array at the bottom of FIG. 6, for the single color — red 
or blue — that needs to be interpolated at a given pixel, 
the interpolation procedure can draw on measurement 
data from all four nearest-neighbor elements and from 
two of the four next-nearest neighbors. For example, 
consider the element shown with the bold outline in 
FIG. 6. It has actual measurements for the luminous 
intensity of green and blue, and requires an 
interpolation to be done to determine the contribution 
that red will make to the composite image. Not only 
do six of the eight nearest- and next-nearest neighbor 
elements contain actual red data, but they also contain 
the red-to-green intensity ratio. In the present 
invention, it will be that data that will take dominance 
if its interpolated value at the element in question 
differs from the straight-forward interpolation of red. 
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That is, the software used to "assemble" the 
composite image is written to ensure that outcome . 

It is submitted that it is readily apparent from this passage of Howell that 
Fig. 6 does not show images including a green image as apparently alleged by 
the Examiner, but that Fig. 6 merely shows arrays of red, blue, and oreen data 
that are combined bv software to generate a composite color image . It is 
submitted that nothing whatsoever in this passage of Howell and Fig. 6 of 
Howell disclose an image processing unit which generates and outputs a single 
monochromatic image using only pixel data detected by photo detectors 
corresponding to the two filters for the selected color of the three colors as 
recited in claim 1 . 

It is submitted that Howell only discloses generating and outputting a 
color image in a single-shot operation, and generating and outputting a 
composite color image in a two-shot operation. 

Accordingly, for the reasons discussed above, it is submitted that Howell 
does not disclose the feature of claim 1 wherein the imaoe processing unit 
generates and outputs a single monochromatic imaoe using only pixel data 
detected bv photo detectors corresponding to the two filters for the s elected 
color of the three colors . 

Independent claim 6 recites an image processing method comprising the 
steps of picking up, with a color imaging device, an image of an object formed 
on an imaging surface by an imaging optical system, the color imaging device 
including a plurality of photo detectors arranged on the imaging surface, each of 
the photo detectors forming a pixel, and a plurality of sets of four filters for 
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three colors, two filters of the four filters being for a selected color of the three 

colors, the filters being arranged at positions respectively corresponding to the 

photo detectors, the image of the object being formed on the photo detectors 

through the filters by the imaging optical system, extracting pixel data of pixels 

corresponding to the two filters for the selected color of the three colors from 

image data of the image picked up, shifting the imaging optical system and the 

color imaging device relative to each other by a distance corresponding to 1/n (n 

is an integer) of a pixel on the imaging surface, picking up, with the color 

imaging device, the image of the object formed on the imaging surface after the 

shifting step, extracting pixel data of the pixels corresponding to the two filters 

for the selected color of the three colors from image data of the image picked up 

after the shifting step, and generating a monochromatic image by synthesizing a 

plurality of pixel data extracted before and after the shifting step . 

In explaining the rejection of claim 6, the Examiner states as follows: 

In regards to claim 6 as being a method for using 
the device claimed in claim 1 see examiners notes on 
the rejection of claim 1 . 

However, it is submitted that Howell does not disclose generating a 

monochromatic image bv synthesizing a plurality of pixel data extracted before 

and after the shifting step as recited in claim 6 for substantially the same 

reasons discussed above that Howell does not disclose the similar feature of 

claim 1 . 

Dependent claim 7 recites an image processing method according to claim 
6, wherein the shifting step, the pickino-up step after the shifting step, and the 
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extracting step after the shifting step are repeated a plurality of times for a 
plurality of different shift positions . 

Claim 7 is directed to the feature of the present invention shown, for 
example, in Fig. 2C wherein a green image is generated bv combining green 
image data obtained at eight different positions arrived at bv performing seven 
shift operations . 

In explaining the rejection of claim 7, the Examiner states as follows: 

In regards to claim 7 Howell discloses a "shifting 
step" as described in the rejection of claim 1 wherein 
in order to take a picture on Howell's camera on the 
"two-shot operation (column 6, line 52, Howell)" the 
camera takes a picture, which entails extracting data 
from the photo detectors, namely the green color 
filter's photo detectors, shifts the light-sensitive array, 
takes a picture, shifts the light-sensitive array back to 
its starting position or in other words "said shifting 
step, said image pickup step and said extraction step 
are executed repeatedly a plurality of times for 
different shift positions." 

However, in Howell's two-shot operation referred to by the Examiner, a 
shifting step, a picking-up step after the shifting step, and an extracting step 
after the shifting step are performed a single time for a single shift position , 
rather than being repeated a plurality of times for a plurality of d ifferent shift 
positions as recited in claim 7, 

Accordingly, for the reasons discussed above, it is submitted that Howell 
does not disclose the feature of claim 7 wherein the shifting step, t he picking-up 
step after the shifting step, and the extracting step after the shift ing step are 
repeated a plurality of times for a plurality of different shift positions . 
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New dependent claim 8 recites an image processing apparatus according 
to claim 1, wherein the image processing unit also generates a color image 
using a plurality of image data picked up before and after the shifting , and 
wherein the image processing apparatus further comprises an output unit for 
selectively outputting one of the single monochromatic image and the color 
image . 

Claim 8 is directed to the feature of the present invention described, for 
example, on page 9, line 27, through page 10, line 4, of the specification. 

It is submitted that Howell only discloses generating and outputting a 
color image in a single-shot operation, and generating and outputting a 
composite color image in a two-shot operation. Accordingly, it is submitted that 
Howell does not disclose an output unit for selectively outputting one of the 
single monochromatic image and the color image as recited in claim 8. 

New independent claim 12 recites an image processing method 
comprising the steps of picking up, with a color imaging device, an image of an 
object formed on an imaging surface by an imaging optical system, extracting 
pixel data of pixels corresponding to a single color from image data of the image 
picked up, shifting the imaging optical system and the color imaging device 
relative to each other, picking up, with the color imaging device, the image of 
the object formed on the imaging surface after the shifting step, extracting pixel 
data of the pixels corresponding to the single color from image data of the 
image picked up after the shifting step, and generating a monochromatic image 
bv synthesizing the pixel data extracted before and after the shifting step . 
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It is submitted that Howell does not disclose generating a mono chromatic 
image bv synthesizing the pixel data extracted before and after the shifting step 
as recited in claim 12 for substantially the same reasons discussed above that 
Howell does not disclose the similar features of claim 1 and 6 . 

New dependent claim 14 recites an image processing method according 
to claim 12, wherein the color imaging device includes a plurality of photo 
detectors arranged on the imaging surface, each of the photo detectors forming 
a pixel, and a plurality of sets of four filters for three colors including the single 
color, two filters of the four filters being for the single color, the filters being 
arranged at positions respectively corresponding to the photo detectors, the 
image of the object being formed on the photo detectors through the filters by 
the imaging optical system, wherein in the shifting step, the imaging optical 
system and the color imaging device are shifted relative to each other by a 
distance corresponding to a predetermined number of pixels on the imaging 
surface, and wherein the generating step generates a single monochromatic 
image using only pixel data detected bv photo detectors corresponding to the 
two filters for the single color . 

It is submitted that Howell does not disclose the feature of claim 14 
wherein the generating step generates a single monochromatic image using only 
pixel data detected bv photo detectors corresponding to the two filters for the 
single color for substantially the same reasons discussed above that Howell 
does not disclose the simitar features of claims 1. 6, and 12 . 
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Since Howell does not disclose the features of claims 1, 6-8, 12, and 14 
discussed above, it is submitted that claims 1, 6-8, 12, and 14 and claims 2-5, 
9-11, and 13 depending from claims 1, 6, and 12 patentably distinguish over 
Howell in the sense of 35 USC 102(a), and it is respectfully requested that the 
rejection of claims 1-7 under 35 USC 102(a) as being anticipated by Howell be 
withdrawn . 

Although dependent claims 2-5, 9-11, and 13 are considered to be 
allowable by virtue of their dependency from allowable independent claims 1, 6, 
and 12, it is noted that these dependent claims also recite further features of 
the present invention which are not seen to be disclosed or suggested by the 
prior art at least in combination with all of the other features recited in 
independent claims 1, 6, and 12 from which they depend . 

As recognized by the Examiner, the other references cited but not relied 
upon neither disclose nor suggest the present invention, and thus no further 
discussion of these other references is deemed necessary at this time. 

It is submitted that all of the Examiner's objections and rejections have 
been overcome, and that the application is now in condition for allowance. 
Reconsideration of the application and an action of a favorable nature are 
respectfully requested. 

To the extent necessary, the applicants petition for an extension of time 
under 37 CFR 1.136. Please charge any shortage in fees due in connection 
with the filing of this paper, including extension of time fees, or credit any 
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overpayment of fees, to the deposit account of Antonelli, Terry, Stout & Kraus, 
LLP, Deposit Account No. 01-2135 (500.40886X00). 

Respectfully submitted, 

ANTONELLI, TERRY, STOUT & KRAUS, LLP 




Melvirf Kraus 
Registration No. 22,466 



MK/RSS 

(703) 312-6600 
Attachments 
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APPENDIX 



VERSION WITH MARKINGS TO SHOW CHANGES MADE 

Changes made to the application by the present amendment are indicated 
below, with brackets indicating deleted matter and underlining indicating added 
matter. 

IN THE CLAIMS 

New claims 8-14 have been added. 
Claims 1-7 have been amended as follows: 

-1. (Amended) An image processing apparatus comprising: 

an imaging optical system for forming an image of an object on an 
imaging surface; 

a color imaging device including 

a plurality of photo detectors arranged on [said] the 
imaging surface, each of [said] the photo detectors forming a 
pixel, and 

a [multiplicity] plurality of sets of four filters for three 
colors, two filters of [said] the four filters being [used] for a 
selected [one] color of [said] the three colors, [said] the 
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filters being arranged at positions respectively corresponding 
to [said] the photo detectors, [respectively,] 

[said] the image of the object [image] being formed on 
[said] the photo detectors through [each of said] the filters 
by [said] the imaging optical system; 
shift drive means for shifting [said] the imaging optical system and 
[said] the photo detectors [relatively] relative to each other by a [length] 
distance corresponding to a predetermined number of pixels [in said] on the 
imaging surface; and 

an image processing unit for generating an image using a plurality 
of image data picked up before and after [said shift] the shifting ; 

wherein [said] the image processing unit generates and outputs a 
single monochromatic image using only [the] pixel data detected by [the] photo 
detectors [having said] corresponding to the two filters [of a single] for the 
selected color of the three colors . 

2. (Amended) An image processing apparatus according to [Claim] claim 1, 
wherein [said] the color filters are arranged according to [the] a Bayer scheme. 

3. (Amended) An image processing apparatus according to [Claim] claim 1, 
wherein [said] the shift drive means shifts [said] the imaging optical system and 
[said] the photo detectors [relatively] relative to each other by a [length] 
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distance corresponding to 1/n (n[:] is an integer) of a pixel on the imaging 
surface . 

4. (Amended) An image processing apparatus according to [Claim] claim 1, 
wherein [said single] the selected color [used by said two filters] of the three 
colors is green. 

5. (Amended) An image processing apparatus according to [Claim] claim 1, 
wherein [said] the image processing unit interpolates [the] pixels lacking [the] 
pixel data of [said single] the selected color of the three colors in [an] one of the 
plurality of image data with [the] pixel data in another one of the plurality of 
image data. 

6. (Amended) An image processing method comprising the steps of: 

picking up A [the] with a color imaging device, an image of an object 
formed on [the] an imaging surface [of] by an imaging optical system, [by a] the 
color imaging device including 

a plurality of photo detectors arranged on the imaging 
surface, each of the photo detectors forming a pixel, and 

a [multiplicity] plurality of sets of four filters for three 
colors, two filters of [which process] the four filters being for 
a selected [one] color of the three [primary] colors, [said] the 
filters being arranged at positions respectively corresponding 
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to [said] the photo detectors, [respectively, on said imaging 
surface;] 

the image of the object being formed on the photo 
detectors through the filters bv the imaging optical system; 
extracting [the] pixel data of [the] pixels corresponding to [said] the 
two filters [of a single] for the selected color of the three colors from [the] 
image data of [said] the image picked up; 

shifting [said] the imaging optical system and [said] the color 
imaging device relative to each other by a [length] distance corresponding to 1/n 
(n[:J is an integer) of a pixel [relatively to each other in said] on the imaging 
surface; 

picking up, [by said] with the color imaging device, the image of 
the object formed on [said] the imaging surface after [said shift] the shifting 
step ; 

extracting [the] pixel data of the pixels corresponding to [said] the 
two filters [of a single] for the selected color of the three colors from [the] 
image data of [said] the image picked up after [said shift] the shifting step : and 

generating a monochromatic image by synthesizing a plurality of 
pixel data extracted before and after [said shift] the shifting step . 

7. (Amended) [A] An image processing method according to [Claim] claim 6, 
wherein [said] the shifting step, [said image pick-up] the picking-up step after 
the shifting step, and [said extraction] the extracting step after the shifting step 
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are [executed repeatedly] repeated a plurality of times for a plurality of different 
shift positions. -- 
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